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ABSTRACT . 

The fair and accurate treatment of multiple test scores for 
law school applicants who take the Law School Admission Test (LSAT) more than 
one time is the focus of this study. The study reexamines the differential 
validity and predictive accuracy of the different test’ scores that are 
presented by repeat test takers. The study includes U.S. .law schools that 
participated in the Law School Admission Council correlation studies in 1996 
and 1997. Data are for 152 schools. The study also includes descriptive data 
comparing repeat test takers with those who only took the LSAT once. The data 
in this and previous studies underscore the need to consider individual 
circumstances when evaluating scores for repeat test takers. Although the 
aggregate statistics confirm that, overall, using the average score for 
repeaters provides more accurate prediction of first-year grades, there are 
individual test takers for whom this is not the case. There are also examples 
in which one of the other score options would provide more accurate 
information about an individual applicant. In some instances, the initial 
score provides the best prediction of first-year law school student 
performance. In other cases, the initial score does not reflect the ability 
of the' test taker accurately, and the test taker has chosen to repeat the 
test to obtain a more accurate representation of his or her ability. If a 
general rule that is most fair to the majority of law school applicants, the 
data continue to support the recommendation to use the average score. 

However, score users need to be sensitive to individual differences. (SLD) 
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Executive Summary 

The fair and accurate treatment of multiple-test scores for law school applicants who take the Law School 
Admission Test (LSAT) more than one time is the focus of this study. The conclusion from earlier research 
studies is that the simple arithmetic average of multiple scores provides the best prediction of subsequent law 
school performance for repeaters. Since the content and scoring of the LSAT have changed since the Wightman 
1990 study, the current study reexamines the differential validity and predictive accuracy of the different test 
scores that are presented by repeat test takers. In particular, the study examines the validity of using the (1) most 
recent, (2) initial, (3) highest, and (4) average score for repeaters. 

The study includes U.S. law schools that participated in the Law School Admission Council (LSAC) 
Correlation Studies in 1996 and 1997. Data for four entering classes, from the fall of 1993 through the fall of 
1996, were combined within each school. The sample contains only the 152 schools whose combined four-year 
enrollment included a total of 50 or more first-year students who had taken the LSAT on more than one 
occasion. Data were combined across four years in order to obtain sample sizes large enough to assure stability 
in the validity estimates. In addition to validity data, the study also provides descriptive data comparing one- 
time test takers with repeat test takers. Repeat test takers tend to earn lower LSAT scores than one-time test 
takers regardless of whether initial, most recent, highest, or average score is considered. One-timers also tend to 
have slightly higher undergraduate and first-year law school grade-point averages. 

A primary concern for LSAT score users with regard to repeat test takers is the question of which of the 
scores or score combinations obtained from repeat test takers will most accurately predict subsequent 
performance in law school. Despite the changes in the LSAT, the results of the replication study support the 
original recommendation to use the arithmetic average of multiple test scores. A primary advantage of the 
average score is that it makes use of all the data that are available about the applicant. Further, no other score 
has been found to be superior to it. The data in this and previous studies also underscore the need to consider 
individual circumstances when evaluating scores for repeat test takers. That is, although the aggregate statistics 
confirm that, overall, using the average score for repeaters provides more accurate prediction of first-year 
grades, there are individual test takers for whom this is not the case. As important, there are examples in which 
one of the other score options would provide more accurate information about an individual applicant. In some 
instances, the initial score provides the best prediction of first-year law school performance. Intervening 
preparation may result in higher scores that overpredict subsequent law school performance. In other instances, 
the initial score does not accurately reflect the ability of the test taker and the test taker self-selected to repeat the 
test in order to obtain a more accurate reflection of his or her ability. 

If a general rule that will be most fair to the majority of law school applicants is to be applied, the data 
continue to support the recommendation of the average score for general use. Regardless, score users need to be 
sensitive to individual differences among test takers and evaluate multiple scores in the context of additional 
information. 



Introduction 

Questions about how to treat multiple test scores for law school applicants who have taken the Law School 
Admission Test (LSAT) on more than one occasion long have been of concern to law school admission 
committees. Wightman (1990) recommends using the arithmetic average of LSAT test scores, and in her study 
concluded that applicants’ first-year law school performance is more accurately predicted by the average of the 
LSAT test scores than by the most recent score, the initial score, or the highest score. The current study 
reevaluates her recommendation using data from the current version of the LSAT (adopted June, 1991), which 
differs in both content and format from previous versions. 

This study is a partial replication of the Wightman (1990) study and is designed to address the following 
questions: 
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1 . Do the traditional predictors of first-year law school performance, LSAT or undergraduate grade- 
point average (UGPA), or the combination of both of these predictors result in differential 
prediction between repeat test takers and one-timers? 

2. If applicants have multiple LSAT scores, is the arithmetic average of test scores more accurate for 
predicting first-year law school performance than the most recent, the initial, or the highest score? 

In addition, the report includes descriptive data comparing one-time test takers with repeat test takers. 

The 1990 repeater study included 13,371 first-year students entering 46 law schools in the fall of 1987. 
Approximately 76% of these students had only one LSAT score, 22% had exactly two scores, and 2% had 
exactly three. The present study uses the same criteria for inclusion that was used in Wightman ( 1 990) of 50 or 
more repeaters per law school. We combined data for students entering law school in the fall of 1993, 1994, 
1995, and 1996. (Starting with 1993 provides sufficient repeater sample size for the current LSAT scale.) 

Methods 



Sample 

The sample used in this study was drawn from the pool of 128,542 U.S. law school students whose records 
were used in the Law School Admission Council (LSAC) Correlation Studies for 176 U.S. law schools in 1996 
and 1997. These students entered law school in the fall of 1993 through the fall of 1996. The data from the four 
classes were combined in order to increase the number of records for repeat test takers, both to assure stability in 
the analyses and to increase the representation of law schools. Overall, there are 1 12,967 students included in 
this study. Approximately 78% of these had only one LSAT score, 20% had exactly two scores, and 2% had 
exactly three. (Only the three most recent scores were retained in the database used.) The percent repeating 
increased slightly across the four years. 

Schools were omitted from the sample if they did not have 50 or more students who had repeated the LSAT 
at least once or if they had changed their method of reporting first-year average during the years analyzed. Data 
were analyzed separately for the 152 law schools included in the sample. On average there were approximately 
167 repeater test takers and 577 one-time test takers represented in each school. The large repeater sample size 
is primarily a result of including four years of student data in the study, when available. Schools participating in 
the LSAC Correlation Studies for the first time may only have provided data for one year. The largest number of 
repeat test takers in a school was 593 out of 1,738 total test takers. One school contained the minimum number 
of repeaters, 50 (out of 466). 

LSAT Version 

All students whose data are used in this study were tested with the version of the LSAT that includes five 35- 
minute sections. One section is a variable section that contains material used to pretest new questions or 
preequate new test forms. The variable section does not contribute to the test taker’s score. The other four 
sections contain items designed to measure reading comprehension, verbal reasoning, and analytical reasoning 
proficiency. The specific item type makeup is as follows: 



Item Type 


Number of Items 


Time 


Number of 
Sections 


Reading Comprehension 


26-28 


35 minutes 


1 


Logical Reasoning 


24-26 


35 minutes 


2 


Analytical Reasoning 


22-24 


35 minutes 


1 



The total number of scored items is usually 101. A single LSAT score derived from the sum of the total 
number of questions answered correctly across the four scored sections is reported on a scale that ranges from 
120 to 180. These scale scores allow comparison of scores achieved on different forms of the test. For example, 

0 




6 



regardless of the form administered, a scaled score of 150 reflects the same level of proficiency. A 30-minute 
writing sample is administered at the end of the test. These samples are sent to all schools to which the test taker 
applies, but are not scored. 

Variables Used in the Study 

The variables analyzed in this study are those that are currently used in the LSAC Correlation Studies: first- 
year average in law school (FYA), undergraduate grade-point average (UGPA), and LSAT score. Only students 
for whom data are available on each of the three variables are included in this study. LSAT score and UGPA are 
the predictor variables. FYA is the criterion variable, or the variable that LSAT and UGPA are used to predict. 

First-year Average. This variable is the average grade earned by the student in the first year of law school. 
First-year average is provided for each student by individual law schools. Different law schools use different 
scales for first-year grades. In order to maintain the confidentiality of the individual schools and to allow direct 
comparison across law schools, the FYA values reported by each school were transformed to a scale having a 
mean of 50 and a standard deviation of 10. Results presented in this report are on the transformed 50/10 scale. 

Undergraduate grade-point average. The average grade earned by each student during his or her 
undergraduate study was computed by the Law School Data Assembly Service (LSDAS) or according to 
LSDAS procedures, following the computing options selected for the undergraduate school the student attended. 
Grades computed in this manner are expressed on a scale of 0.00 to 4.33. The UGPAs used in these studies were 
the same as those used in the correlation studies carried out for individual law schools. 

LSAT scores. Four different LSAT scores were analyzed for repeat test takers as part of this study: (1) the 
most recent LSAT score earned by the repeater, (2) the initial score of the two or three (three is the maximum 
number considered in this study), (3) the highest of the two or three scores, and (4) the average of the two or 
three scores. Only LSAT scores reported on the 120 to 180 score scale were used in this study. 

Analysis Methods 

A primary focus of this study is the impact on predictive validity of using different scores for LSAT 
repeaters. That is, what LSAT score for repeaters most accurately predicts their subsequent performance in law 
school? The same analyses that are used in the ongoing predictive validity studies for individual schools that 
participate in the LSAC Correlation Studies are used in this study. That is, least-squares regression analysis is 
used to predict first-year average from undergraduate grade-point average and the various LSAT scores for 
repeaters. The analyses are carried out separately by law school using the pooled four-year data. Adjustments 
are not made for differences among undergraduate institutions when combining across undergraduate schools to 
analyze undergraduate grade-point averages. The acceptability of unadjusted undergraduate grades is supported 
by the findings reported by Rock and Evans (1982) that (1) much of the gain in prediction disappears when the 
adjusted grades are used in conjunction with the LSAT to predict FYA and (2) schools for which adjustment 
was successful in one year were not necessarily those for which adjustment was successful in a subsequent year. 

For students who presented only one LSAT score, the data are the same for each comparison. For repeaters, 
each analysis is based on a different LSAT score: the most recent, the initial, the highest, or the average. 

Results 

The results from this study are presented in two parts. The first part includes descriptive data about the one- 
time and repeat test takers. The results of applying the various prediction equations derived using the total group 
data (both one-time test takers and repeaters) to repeat test takers are reported in the second part. 

Descriptive Statistics 

Descriptive statistics for the sample of students within the law schools used in this study are presented in 
Table 1 and Figures 1-5. These data provide summary information about the number and proportion of one-time 
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and repeat test takers among the first-year law students in the schools included in this study. This allows for the 
comparison of LSAT performance, undergraduate grade-point average, and performance in first-year of law 
school between one-time test takers and repeat test takers. 

Table 1 provides information about the prevalence of repeat test takers in the entering classes included in the 
1996-97 LSAC Correlation Studies. For the pooled data, over 22% of the students have taken the LSAT more 
than one time. Overall, the percentage of repeaters increased over the four years covered by this study. For the 
1993-94 entering class data only 20.6% of the students have taken the LSAT more than one time. The last 
entering class included (1996-97) had 24.5% repeat test takers. Recall that schools were omitted from the 
current study if they did not have at least 50 repeat test takers or had changed their method of reporting first-year 
average during the years analyzed. 

TABLE l 

Number and percentage of one-time and repeat LSAT test takers among schools included 
in the 1997 LSAC Correlation Studies 



Entering 

Class 


Total 


Number of 


Repeaters 


One-timers 


Schools 


N 


% 


N 


% 


1993-94 


28,164 


148 


5,814 


20.6 


22,350 


79.4 


1994-95 


29,841 


151 


6,200 


20.8 


23,641 


79.2 


1995-96 


27,886 


148 


6,691 


24.0 


21,195 


76.0 


1996-97 


27,076 


149 


6,625 


24.5 


20,451 


75.5 


Pooled Data 


112,967 


152 


25,330 


22.4 


87,637 


77.6 



The current study investigates repeat test takers performance at 152 U.S. law schools. This number is more 
than three times the 42 law schools included in the previous Wightman (1990) study. The act of pooling data 
across the four years contributed to the increased sample size. Although the percentage of repeaters increased 
over the years studied, the combined repeater percentage (22%) is comparable to the 24% repeaters found in the 
sample used in 1990. Figure 1 presents the distribution over the 152 law schools of the percentage of repeat test 
takers at each law school. Repeaters comprised between 6% to 53% of the entering classes studied. 





FIGURE 1 . Distribution of percentage of repeat test takers at the 
law schools included in this study. 
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Figure 2 shows the distribution of law school LSAT means for one-timers and repeaters. For the repeat test 
takers the most recent, initial, highest, and average test scores are included. In calculating the data presented in 
this figure, the mean LSAT score was calculated separately for the one-time test takers and the repeat test takers 
at each school. (LSAT scores are reported on a scale of 120 to 180.) The distributions demonstrate that repeat 
test takers tend to be a lower scoring group than the one-time test takers. In general, one-time test takers tend to 
earn higher LSAT scores than repeat test takers regardless of whether the most recent, initial, highest, or average 
score is considered for the repeaters. The range of mean LSAT scores across schools is fairly substantial, 
varying from a low mean of 132.5 to a high mean of 169. The size of the range is larger for the repeaters when 
the highest LSAT score is used. The student-level means and standard deviations are shown in Table 2. Once 
again, the average score is higher for the one-time test takers than for any score reported for the repeaters. As 
expected, the least difference is found between the one-time test takers’ mean and the mean for the highest 
reported score for the repeat test takers. 
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FIGURE 2. Distribution of LSAT mean for one-time and repeat test takers 
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TABLE 2 

Means and standard deviations of LSAT test scores for one-time and repeat test takers 



Score 


Mean 


Standard Deviation 


Repeaters: Most Recent 


153.6 


7.0 


Repeaters: Initial 


149.7 


6.4 


Repeaters: Highest 


154.2 


6.6 


Repeaters: Average 


151.9 


6.3 


One-Timers Test Score 


157.5 


7.1 



It is worth noting that repeaters are a self-selected group who, for the most part, choose to repeat the test 
because they believe the initial score reflects an ability lower than their true ability. Therefore, this argument 
might explain why the average score-change for repeaters who became first-year law school students is an 
increase of about three score points. These data are consistent with previously reported score gains (Wightman, 
1990; Pitcher, 1977). 

Despite the fact that repeat test takers, on average, increase their LSAT performance, even their increased 
scores tend to be lower than those earned by one-time test takers at their law school. Figure 3 shows the 
difference between each school’s mean score earned by one-time test takers and the mean scores earned by 
repeat test takers. In general, the comparisons show that within each law school, regardless of the score reported 
for the repeaters, the one-timers have higher LSAT mean scores. The magnitude of the difference varies among 
individual schools, but only two schools showed a mean for repeaters (one when the highest score was reported 
and one when the most recent score was reported) that exceeded the one-timers’ mean score. 
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LSAT School Mean for One-timers Minus LSAT School Mean for Repeaters 



FIGURE 3. Distribution of test score differences. LSAT mean for one-time test 
takers minus the LSAT mean for repeat test takers using the multiple scores. 



The mean undergraduate grade-point average is similar for the two groups. Figure 4 compares the average 
UGPA for one-time test takers versus repeat test takers at each school. (UGPAs are expressed on a scale of 0.00 
to 4.33.) The distribution shows the UGPA school mean for one-time test takers minus the UGPA school mean 
for repeat test takers. The distribution shows a slight positive skew, but the differences are small. The maximum 
difference is only 0. 15 grade point. The overall mean and standard deviation of UGPA for one-time and repeat 
test takers is also shown in Figure 4. The one-time test takers have an average of 3.24 and the repeat test takers 
have an average UGPA of 3.15. The two groups sampled have comparable UGPA variance (0.4). The mean 
UGPA for one-timers is about one-fifth of a standard deviation higher than the mean UGPA for repeaters. 
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UGPA School Mean for One-Timers Minus UGPA School Mean for Repeaters 



FIGURE 4. Distribution of undergraduate grade-point average differences. School 
UGPA mean for one-time test takers minus UGPA mean for repeat test takers. 



Figure 5 shows the FYA school mean for one-timers minus FYA school mean for repeaters. The largest 
difference in the repeaters’ favor is 1.5 grade-points. The distribution shows a slight positive skew with most of 
the distribution’s area above zero, indicating that the one-timers on average earn slightly higher FYA’s than 
their classmates who took the LSAT more than one time. In only 4 of the 152 schools examined in this study 
does the mean first-year average for repeaters exceed the mean for one-time takers. The overall FYA means are 
50.49 for repeaters and 48.31 for one-time test takers. The standard deviation for both groups is about 10 grade- 
points. The mean FYA for one-timers is about one-fifth of a standard deviation higher than the mean FYA for 
repeaters. 
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FYA School Mean for One-Timers Minus FYA School Mean for Repeaters 



FIGURE 5. Distribution of first-year grade-point average differences. School FYA 
mean for one-time test takers minus FYA mean for repeat test takers. 



These results are not fully consistent with those reported by Wightman (1990). The repeaters in her study 
achieved lower grades in law school than did their nonrepeater counterparts, but earned comparable 
undergraduate grade-point averages. 

Predicting First-Year Averages 

A primary research question addressed by this study was whether or not LSAT, UGPA, or the combination of 
these two predictor variables differentially predicted FYA for those law school students that took the LSAT 
more than once relative to those who took the LSAT only one time. Three separate predictor combinations are 
analyzed for these data: LSAT alone, UGPA alone, and LSAT and UGPA combined. The LSAT score (or 
combination of scores) used for the repeat test takers was varied for each expression that used LSAT score as a 
predictor resulting in nine total regressions. Each prediction equation is derived using the total group data within 
each law school (repeaters and one-time test takers combined) and then applied to the groups separately. 

Comparisons between the predicted and actual FYA were made for repeaters and one-timers based on each 
regression equation. The calculations and comparisons were made using each school’s own grading scale, but all 
of the first-year averages were converted to a scale where the mean for each school was set to 50 and the 
standard deviation to 10. The conversion was made to preserve the confidentiality of the data and to allow 
comparisons across law schools. 

Figures 6-8 show the differences between predicted and actual first-year average means for one-timers and 
repeaters using the nine regression equations. Note that a negative value indicates that the regression equation 
underpredicted the mean performance of a subgroup in a law school, while a positive difference indicates that 
the regression equation overpredicted the mean performance of a subgroup in a law school. 



ERIC 



13 



BEST COPY AVAILABLE 



10 



Figures 6a through 6d display the distributions of differences between predicted and actual First-year average 
means for one-timers and repeaters using only LSAT as the predictor variable. The regression equations using 
the most recent LSAT score and the highest LSAT score for the repeaters produced the distributions displayed 
in Figure 6a and Figure 6c, respectively. The positive skew in the repeaters’ distribution indicates 
overprediction. The one-timers’ performance is slightly underpredicted by the same regression equation. When 
the initial LSAT score is used for the repeaters, the opposite pattern appears as shown by the data presented in 
Figure 6b. The regression equation based on these data from the repeaters and one-timers combined tends to 
underpredict the first-year law school performance for the repeaters and slightly overpredict the one-timers’ 
performance. Figure 6d reveals that the most accurate use of the LSAT score is found by assigning the average 
LSAT score for the repeaters. 
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FIGURE 6. Frequency distributions of differences between predicted and 
actual FYA means for one-timers and repeaters at participating law 
schools using LSAT as the predictor variable (using the multiple scores for 
the repeaters). 
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Figure 7 contains the differences between actual and estimated FYA based on a regression equation using 
only undergraduate grade-point average. The distributions reveal that UGPA alone overpredicts the performance 
of repeat test takers to a greater extent than LSAT alone, but does not perform badly as a predictor of one-time 
test takers’ law school performance. 
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FIGURE 7. Frequency distributions of differences between predicted and actual 
FYA means for one-timers and repeaters at participating law schools using UGPA as 
the predictor variable. 



The model that accounts for the most variance in first-year averages contains both LSAT and UGPA. This 
model accounts for more of the variance in the first-year averages than either model containing only LSAT or 
only UGPA . Figures 8a through 8d show the distribution of differences when varying the LSAT score reported 
for the repeat test takers. The results for the regression equation based on the most recent score and the highest 
score combined with UGPA (Figure 8a and Figure 8c) reveal a similar pattern to the results observed for LSAT 
alone. In both cases, the first-year performance is over predicted for the repeat test takers. When the initial 
LSAT score is used for the repeaters in the predictor combination, the repeaters are underpredicted as shown by 
the data presented in Figure 8b. Once again, the most accurate use of the LSAT score is found by assigning the 
average LSAT score to the repeaters. Figure 8d reveals that LSAT score combined with UGPA (using the 
average LSAT for the repeaters) emerges as a more accurate predictor of law school performance for both 
groups. 
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FIGURE 8. Frequency distribution of differences between predicted and actual 
FYA means for one-timers and repeaters at participating law schools using LSAT 
and UGPA as the predictor variables (using the multiple scores for the repeaters). 



Summary and Discussion 



This study analyzed data from 152 law schools, each of which enrolled 50 or more first-year students who 
took the LSAT on more than one occasion. The present study was conducted to determine whether evidence 
exists of differential prediction among various combinations of LSAT scores, for test takers with multiple LSAT 
scores. 

Regression equations derived by combining data for both one-time and repeat test takers included in this 
study were used to evaluate differential prediction of law school first-year grade-point average when LSAT 
score alone, undergraduate grade-point average alone, or the combination of LSAT and undergraduate grades 
were used as predictors. Each of the multiple scores reported for repeat test takers was used in the predictor 
combinations. The use of undergraduate grades seems to produce the most differential prediction for the repeat 
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test takers studied. Further, the data confirm that using the average LSAT score for the repeat test takers 
produces the least amount of differential prediction. Average LSAT either alone or in combination with UGPA 
produces the most accurate prediction. These conclusions confirm previous recommendations. 

It should be noted that differential prediction is only one aspect of an overall construct validity evaluation. 
Other aspects, such as the strength of the correlation between a predictor and a criterion variable, should also be 
considered when deciding whether prediction equations are equitable and valid. Such studies are also carried out 
on a regular basis (see Roussos & Norton, 1998). 

If a general rule that will be most fair to the majority of law school applicants is to be applied, the data 
support the recommendation of using the average score. Regardless, score users need to be sensitive to 
individual circumstances and evaluate multiple scores in the context of additional information. If there is 
information indicating that a different score more accurately reflects a test taker’s ability (for example, if he or 
she was sick during the first administration), such information should be considered. 
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